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ABSTRACT 

E-learning has become an increasingly important teaching and learning mode in educational institutions and 
corporatetraining. The evaluation of E-learning, however, is essential for the quality assurance of E-learning courses. This 
paper constructs a three-phase evaluation model for E-learning courses, which includes development, process, and 
product evaluation, called the DPP evaluation model. Development evaluation includes, course material design, E- 
learning platform, course Web site design, learning resource, interactivity, assessment, and tutor support. Process 
evaluation includes technical support, Web site utilization, learning interaction, learning evaluation, learning support, 
and flexibility. Product evaluation includes student satisfaction, teaching effectiveness, learning effectiveness, and 
sustainability. This research uses the DPP model to evaluate a purely E-learning course in Distance and open Learning, 
developed by Philadelphia University. According to summative evaluation through a student E-learning experience 
survey, the majority of students were satisfied on all E-learning dimensions of an E-learning course. The majority of 
students thought that the learning effectiveness of this course was equivalent, even better, than face-to-face learning 
because of cross-border collaborative learning, sufficient learning support, and learning flexibility. This study shows that a 
high quality of teaching and learning might be assured by using the systematic DPP evaluation procedure. It is hoped 
that the DPP evaluation model can provide a benchmark for establishing a wider E-learning quality assurance 
mechanism in educational institutions. 

Keywords: E-Learning, QualilyAssurance, Evaluation, Moodle, Learning Management System (LMS), E-learning Platform, 
Pedagogical Model. 


INTRODUCTION 

E-learning has become widely used in conventional 
education, continuing education, and corporate training 
because of its flexibility, richness, and cost-effectiveness. 
United Nations Educational, Scientific and Cultural 
Organization (UNESCO) statistics show that over 455 million 
people around the world received education and training 
through the Internet in 2008, UNESCO Institute for Statistics 
(2011). Over 70% of universities in the USA were providing E- 
learning courses, and more than 6.1 million university 
students were taking at least one e-learning course, which 
accounted for over 31 % of the total number of university 
students in the USA, Allen and Seaman (2011). 

With the rapid development of information technology, 
student attitudes towards E-learning are becoming more 
positive. According to the Survey on the Demand for 
Continuing Education in Hong Kong 2007-2008, 40.4% of 
the respondents showed positive attitudes to e-learning 


when they considered pursuing a continuing education 
program, while 83.9% of them hoped to try the blended 
learning approach. 

With the rapid development of e-learning, there is also an 
increasing interest in e-learning research. Among all the 
research topics, quality assurance of e-learning has 
attracted the greatest concern. Jung, Wong, Cheng, 
Baigaltugs and Belawati (2011) found that various national, 
regional, and international initiatives have been 
undertaken with regard to quality assurance in E-learning. 
Endean, Bai, and Dui (2007) stated that those concerned 
about online learning have been developing and 
publishing ideas for over a decade about how to manage 
the quality of the learning experience of those trying to 
study through the Internet. 

However, Jung and Latchem (2007) found that most 
institutions apply the same quality criteria for E-learning as 
for the other modes of delivery. Endean, Bai, and Dui (2010) 
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pointed that new entrants to the e-learning field were 
unlikely to have existing internal procedures to cover quality 
assurance of this new mode of operation. 

Course quality is assured by a series of evaluations, and e- 
learning should be no exception. In this paper, the author 
proposes an e-learning course evaluation model, called 
DPP model, for quality assurance and analyzes its concrete 
application using an E-learning course developed by 
Philadelphia University-Jordan. 

Construction of an E-learning Course Evaluation Model 

Referring to the real experience in producing e-learning 
courses at Philadelphia University, the author proposes a 
system for evaluating E-learning course that consists of 
three evaluation activities: development evaluation, 
process evaluation, and product evaluation; in short, the 
DPP model. Based upon the proposed DPP model and in 
line with the components and E-learning characteristics, 
the e-learning evaluation model consists of 17 items as 
shown in Table 1. 

Development evaluation of E-learning courses involves 
analyzing every component of course development, 
including the course material design. E-learning platform, 
course website design, learning resource, student-student 
interactions, Assessment, and Tutors. Evaluating of e- 
learning teaching process should include the following six 
dimensions: technical support, website utilization, learning 
interaction, learning evaluation, learning support, and 
flexibility. Product evaluation measures the learners' 
satisfaction, teaching effectiveness, learning effectiveness, 
and sustainability- which depend on the results of the 
abovementioned analyses. 

Development Evaluation 


The first step in e-learning development evaluation is to 
analyze the course material design according to the 
Pedagogical Model used in E-learning course 


Development Evaluation 

Process Evaluation 

Product Evaluation 

Course material design 
E-learning platform 

Course website design 
Learning resources 
interactivity 

Assessment 

Tutors 

Technical support 
Website utilization 
Learning interaction 
Learning evaluation 
Learning support 
Flexibility 

Student satisfaction 
Teaching effectiveness 
Learning effectiveness 
Sustainability 


Table 1. The DPP Evaluation Model for E-Learning Courses 


development, which is compiled and prepared by the 
Faculty Council. The Course material design deals with the 
formation of the course team and its members' roles, 
course background, course introduction, course 
objectives, learner analysis, requirements for learning 
facilities and skills, course modules/units, learning materials, 
assessment and examination, communication and 
collaboration in learning, learner support services, 
teaching model, course materials writing schedule, and 
quality assurance. 

When the course material design "Pedagogical Model" 
evaluation has been completed, it is followed by learning 
resources, interactivity, assessments, analyzing the 
construction of the E-learning platform, course website, 
and the training of tutorial staff, which is implemented by 
the deanship of distance learning at the Philadelphia 
University. Table 2 lists the e-learning course development 
and evaluation steps. 

Since this course was developed by the Deanship of 
Distance Learning at Philadelphia University-Jordan, the 
evaluation of the material design, learning resources, 
interactivity, and assessments procedures, followed by the 
course development procedures of the Deanship of 
Distance Learning at Philadelphia University-Jordan. 
Outside experts were invited to do an external evaluation. 
The requirements for external experts were professors in 
education with at least 8 years experience in distance 
education research and teaching. 

The principle of six types of interactions was also 
emphasized, including the interactions between students 
and interface, between students and teachers, among 
students, between students and learning content, between 


Course content 

Responsible Organization 

Evaluation 

Course material design 
"Pedagogical Model". 
Learning resources 
Interactivity 

Diagnostic and Formative 
Assessments 

Deanship of Distance 
Learning 

Philadelphia University 
Jordan 

Course development 
team "Faculty 

Council And Quality 
Assurance team"; 

Distance Learning 
Council. 

Using E-Learning platform 
“Moodle" 

Course Website 

Tutors Training 

Avicenna center for E- 
Learning 

Philadelphia University 
Jordan 

Avicenna Center for 
E-learning; 

Multimedia designer 
and producer; 

Module external 

assessor. 


Table 2. E-Learning Course Development and Evaluation 
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students and learning objectives, between students and 
multimedia learning resources, and between students and 
time management, Zhang (2009), 

The Moodle e-learning platform was used on Learning 
Management System (LMS), The LMS functions could be 
classified into five categories: course content functions, 
communication and collaboration functions, feedback 
and evaluation functions, assignment and assessment 
functions, and administration and management functions, 
Process Evaluation 

Process evaluation refers to evaluating the process of 
course delivery, including the technical support, website 
utilization, learning interaction, learner support, and 
flexibility. Process evaluation mainly uses three 
approaches: survey of students' learning experience and 
feedback; survey of tutors' opinions; and the deanship of 
distance learning council's monitoring of the e-learning 
tutorials. 

The approach to understanding students' learning 
experience and feedback is as follows: establishing a 
special feedback area on the course website, establishing 
email communication between tutors and students, and 
internal reviewer's interviews with tutors and students. For 
example, the students were asked to familiarize themselves 
with various functions of the e-learning platform in the first 
week, referring to the course website guidance. The 
students needed to report to their tutor their degree of 
familiarity and time spent for this purpose. It was found that 
all the students learnt to use this course platform in three to 
five hours. 

In the middle of this course, the reviewers and the deanship 
of distance learning council conducted formative 
evaluation. At the fourth week of this course, individual 
interviews were conducted and a virtual classroom was 
organized for evaluators to gather students' learning 
feedback, including overall evaluation, learning 
experiences, difficulties, and suggestions so that timely 
adjustments could be made. 

Monitoring e-learning tutorials is one of the most important 
tasks for the distance learning council, who needs to log 
into the course website at least once every two weeks to 
observe students' learning progress and difficulties. If 


students' questions are not answered promptly or only 
ambiguous answers are provided, or if there are not many 
posts in the discussion forum area, the distance learning 
council would take immediate action to contact the tutors 
and solve the problem and inspire student's learning 
enthusiasm. 

Process evaluation is an accurate process, which involves 
continuous evaluation throughout the course. Tutors need 
to plan carefully to maintain students' learning enthusiasm 
and help them achieve the final learning objectives. 

Product Evaluation 

Product evaluation of an e-learning course is mainly 
conducted through quantitative research, supplemented 
with students' feedback and suggestions. For the first intake 
of the proposed E-learning course, the online questionnaire 
method was used and all 80 students were surveyed. Sixty 
valid data sets were received; the response rate was 75%. 

Tables 3 to 5 shows the students' evaluation of course 
effectiveness, teaching effectiveness, and learning 
effectiveness. Table 3 shows the results of overall feedback 
on course effectiveness; such feedback is required for all E- 
learning courses at Philadelphia University. The results of 
students' evaluation of various E-learning components of 
the course are shown in Tables 4 & 5. 

In order to understand these results relative to those of the 
face-to-face teaching mode, we adopted the evaluation 
statistics labels used for face-to-face teaching in 
Philadelphia University and calculated the average 
percentage of each item in the questionnaire survey on a 
Likert-type scale, Norman (2010). The scale of the grades is 
explained as follows: 0% - 49.9%, Fail; 50.0% - 59,9%, 
Pass; 60% - 67.9%, Satisfied; 68% - 75.9%, Good; 76% - 
83.9%, Very Good; 84% -100%, Excellent. 


Overall 

Feedback 

Strongly Agree 
Agree 

Un 

decided 

Disagree 

Strongly 

Unsatisfied 

Average 

The course has 
been effective in 
helping me learn. 

21% 

70% 

9% 

0% 

0% 

85% 

The teacher has 
been effective in 
helping me learn. 

48% 

46% 

6% 

0% 

0% 

8 

Attending the 
course has been 
worthwhile. 

80% 

18% 

2% 

0% 

0% 

93% 


Table 3. Overall Feedback on Course Effectiveness 
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Table 3 shows that students' evaluation of learning 
effectiveness, teaching effectiveness, and course worth 
reached the level of "Excellent" using the same evaluation 
statistics method for face-to-face teaching at Philadelphia 
University, 

It can be seen in Tables 4 and 5 that student evaluation of 
degree of satisfaction, teaching products and learning 
products in e-learning components reached the level of 
"Excellent" based on the course evaluation criteria used at 
Philadelphia University. Table 4 shows that satisfaction with 
tutors was highest with an average score of 90.4, but 
interactivity was lowest with an average score of 77,8. In this 
course, the tutors were required to answer student 
questions within 48 hours and it was very much 
appreciated by the students, However, there were no 
requirements for students to respond to other students' 
enquiries in discussion forums. Therefore, interaction 
between students was not so active compared with the 
interaction between tutors and students. 



Strongly Satisfied 

Un 

Un 

Strongly 

Average 


Satisfied 


decided 

satisfied Unsatisfied 


Web site design 

31% 

49% 

12% 

8% 

8% 

78.6% 

Video material 

59% 

41% 

0% 

0% 

0% 

89.3% 

Tutors 

70% 

30% 

0% 

0% 

0% 

90.4% 

Interactivity 

29% 

59% 

12% 

0% 

0% 

77.80% 

E-learning study 
sessions 

38% 

57 % 

5% 

0% 

0% 

80.5% 

Flexibility of learning 

47% 

43% 

10% 

0% 

0% 

85.7% 

Assessment 

44% 

45% 

11% 

0% 

0% 

79.6% 

E-learning 

environment 

40% 

47% 

13% 

0% 

0% 

81% 

Course quality 

46% 

50% 

4% 

0% 

0% 

83% 

Table 4. Students' Degree of Satisfaction with the 



E-learning Course (%) 




Strongly Agree 

Un 

Disagree 

Strongly 

Average 


Agree 


decided 


Disagree 


Teaching pace was 
appropriate video 
program 

51% 

40% 

9% 

0% 

0% 

87.7% 

The lecture was 
knowledgeable 
about the subject 

77% 

23% 

0% 

0% 

0% 

94.9% 

The tutor was 
knowledgeable 
about subject 

660% 

24% 

10% 

0% 

0% 

88.7% 

The tutor explains 
clearly 

69% 

30% 

1% 

0% 

0% 

85.2% 

The tutor has 
been effective 
in helping me to 
learn 

38% 

55% 

7% 

0% 

0% 

81.6% 


Table 5. Students' Evaluation of Teaching Effectiveness (%) 


Conclusion 

The DPP model for evaluating E-learning courses was 
designed and proposed based upon the real experience 
in producing E-learning courses at Philadelphia University 
evaluation model (i.e., development evaluation, process 
evaluation, and product evaluation.) In line with the 
characteristics and process of E-learning teaching and 
learning, 1 7 evaluation items were identified within the DPP 
model. Using the DPP model, the author took an E-learning 
course, which was produced by Philadelphia University as a 
case study to describe and analyze the series of evaluation 
activities. The research results show that this DPP evaluation 
model could effectively ensure the quality of the E-learning 
course in terms of both teaching and learning 
effectiveness. However, the use of the DPP model in this 
study measures only one purely E-learning course and 
further studies are needed. The author hope that this model 
could be one reference in establishing E-learning quality 
assurance models for other educational institutions. 
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